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This paper is one in a foundational research series for the Postsecondary Value Commission authored 
in summer 2019 by scholars with diverse backgrounds and expertise. The research presented in 
these papers applies an equity lens to the philosophical, measurement, and policy considerations and 
assumptions underlying key components of postsecondary value to students and society, including 
investment, economic and non-economic returns, mobility, and racial and socioeconomic justice. 


The Postsecondary Value Commission consulted this foundational research as it developed a conceptual 
definition of postsecondary value, a framework for measuring how institutions and programs create value 
and ensure equitable outcomes, and an action agenda with recommendations for applying the definition 
and framework to change policies and practices. Through this breadth of scholarship, the commission 
was better able to define the value of postsecondary education and the role institutions can play in 
creating a more equitable and fair United States. 


Following the May 2021 release of the commission’s findings, these foundational papers were prepared 
for publication. The views and opinions expressed in these papers do not necessarily reflect the positions 
of individual members of the Postsecondary Value Commission or the organizations they represent. 


The Postsecondary Value Commission along with the Bill & Melinda Gates Foundation and Institute for 
Higher Education Policy are deeply grateful to the authors of this series. The authors’ extensive expertise 
and thoughtful engagement in this work provided the foundation for the commission to develop an 
informed, innovative, and equity-driven framework. They also thank Deborah Seymour for editing the 
written products and the team at GMMB for their creative design and layout. 


Suggested citation: Miller, B. (2021). Measuring Return on Investment for Postsecondary Education: The Case for Using Earnings and Loan 
Repayment Metrics. Postsecondary Value Commission. 
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This paper examines indicators related to earnings and student loan debt for evaluating return on 
investment (ROI) for postsecondary education. It first considers four different options for measuring 
earnings, arguing that measures of postsecondary value should include investment-to-earnings and 
earnings growth metrics. The paper then discusses how earnings should be measured at the program 
and institutional levels. Student loan debt metrics are examined next, with preference for a status-based 
over balance-based repayment rate, followed by a discussion on repayment rates by race/ethnicity, 
gender, and completion status. The paper concludes by discussing the challenges of using publicly 
available data to examine earnings and repayment rates as well as areas for future research.? 


OPTIONS FOR USING EARNINGS TO MEASURE 
POSTSECONDARY VALUE 


While there are a number of ways to measure earnings in the context of postsecondary value (Table 
Al), this paper examines the four most common ways: (1) mean or median earnings levels, (2) the 
share of graduates meeting a baseline earnings threshold, (3) investment-to-earnings ratios, and (4) 
earnings growth. This section explores each of these options and recommends using the third and 
fourth to measure ROI. 


Mean or Median Earnings Levels 


Measures of earnings levels reflect mean or median earnings of graduates without any further 
contextual information. While this straightforward calculation is intuitive, it can also be misleading. 


There are a few challenges with using mean or median earnings levels on their own as a measure of 
value. First, simply looking at the programs with the highest mean or median earnings will draw attention 
to the programs with the best pay—even if there are many others that still provide a meaningful return 
that grants students access to a middle-class life. Second, mean or median earnings do not reflect the 
costs of postsecondary education to students. This is especially problematic for comparing graduates in 
different institutions in a particular field that have similar earnings levels but had different postsecondary 
costs.' Finally, mean or median earnings are also going to be affected by regional variation, which 

can make places that tend to have graduates who move to expensive urban areas look more valuable 
without producing a better return. Because of the issues associated with this metric, this paper does not 
recommend mean or median earnings as a standalone measure of postsecondary value. 


Share of Graduates Meeting a Baseline Earnings Threshold 


Earnings threshold measures capture how many students earn more than a baseline minimum—such 
as $25,000 or $28,000 (the benchmarks used in the College Scorecard to approximate earnings 

of a high school graduate with no college experience) or 150 percent of the poverty level ($18,735 

in 2019). Where to set the benchmark depends on the purpose of the measure. An aspirational 
measure designed to encourage programs to improve should pick a higher bar, such as median 
earnings for graduates at that credential level nationwide. This makes it possible to see how graduate 
earnings from a particular program compare to those of similar students from elsewhere. 


a Miller authored this paper in summer 2019 while serving as vice president of postsecondary education at the Center for American Progress. 
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By contrast, the minimum acceptable earnings thresholds used for accountability purposes should 
use a lower bar tied to the poverty level as a way of assessing that college attendees are at the 
very least not impoverished. This can be used to ensure that graduates’ earnings will be above the 
poverty level and that they will earn more than someone who never attended. 


College Scorecard data for the 2002-03 and 2003-04 cohorts displayed in Table 1 show that most 
institutions have typical earnings above either 150 percent of the poverty level or $28,000 at 10 years 
after entry (results are similar at six and eight years, so they are not presented). The data show that a 
test based on median earnings is tougher to pass than one relying on mean. Given that these results 
include non-completers it is likely that a test based only on graduates would have even higher passage 
rates. There are no good federal data on earnings for students who did not receive federal aid. 


Table 1. Share of Institutions with Mean or Median Earnings Compared to Minimum Baselines 


| Mean€Earnings —_—| | Mean€Earnings —_—| | Median Earnings —_| | Median Earnings —_| 
Public 4-year 100% 99% 100% 98% 
For-Profit 4-Year 67 100% 97% 100% 91% 
Private Nonprofit 4-Year 1,101 100% 96% 99% 92% 
Public 2-Year 688 100% 92% 99% 71% 
For-Profit 2-Year 189 99% 59% 97% 50% 
Private Nonprofit 2-Year 108 99% 78% 95% 66% 
Public Less Than 2-Year 487 100% 711% 99% 52% 
For-Profit Less Than 2-Year 893 90% 27% 67% 19% 
Private Nonprofit Less Than 2-Year 87 97% 60% 83% 51% 
Historically Black Colleges and 94 100% 73% 99% 55% 
Universities (HBCU) 
Predominantly Black Institutions (PBI) 72 100% 12% 100% 42% 
Alaska Native and Native Hawaiian ) 100% 100% 100% 89% 
Serving Institutions (ANNHI) 
Tribal Colleges and Universities (TCU) 28 100% 7% 64% 4% 
Native American Serving, Nontribal 22 100% 91% 100% 73% 
Institutions (NASNT]I) 
Asian American and Native American 56 100% 91% 98% 88% 
Pacific Islander-Serving Institutions 
(AANAPISI) 
Hispanic Serving Institutions (HSI) 339 100% 93% 98% 80% 


Note: Results 10 years after starting college for 2002-03 and 2003-04 entrants who received federal aid. Excludes 
students who are not working or who are enrolled in college at measurement point in time. 


Source: Author analysis of data from the College Scorecard. 


One downside of earnings benchmarks is that they may be set too low or too high. In addition, much 
like earnings levels, benchmarks ignore students’ investment in postsecondary education. Further, 
thresholds often fail to acknowledge that the poverty level changes based on household size. Due to 
these challenges, this metric is not recommended for measuring value. 


Investment-to-Earnings Ratios 


This metric determines whether graduates’ earnings justify their investment in a program. The 
Department of Education’s (ED) gainful employment regulation was the first to operationalize this 
concept at the federal level, by incorporating debt-to-earnings ratios into the standards career 
programs must meet. Debt payments for graduates of these programs must be less than or equal 
to 8 percent of their annual earnings or 20 percent of their discretionary income (earnings over 150 
percent of the poverty level for a single individual).° 


A broader measure of investment-to-earnings would include the total student investment in their college 
education, regardless of whether or how much a student borrowed. This includes federal and state 
grants, work-study, private scholarships, loans, and out-of-pocket spending. Institutional grants and 
discounts should not be included because these do not always represent an actual cash transfer to the 
student. Nor should state spending on operating support since the goal is looking at the price faced by 
the student, not the total cost of delivering an education, which can and must be subsidized in many 
situations. This acknowledges that all forms of investment matter for assessing value, not just loans. 


There are challenges in constructing an investment-to-earnings measure. One is how to compare a 
one-time investment like an out-of-pocket tuition payment to a loan. It is easy to judge the long-term 
cost of a loan because we can calculate what the monthly payment would be and the duration of 
payments. Looking at the long-term cost of an out-of-pocket tuition payment would require making 
assumptions about what would happen to that money if it were invested in other purposes for 
some period of time. Another issue is how to handle amounts paid for non-academic expenses, 
such as room and board. Colleges sometimes control these expenses—as is the case for dorms or 
cafeterias—but other expenses, such as transportation, may be outside of the institution’s control. 

In addition, students would face some of these expenses regardless of college enrollment. It may 
be necessary to exempt a base amount of living expenses from the calculation to reflect costs 
students would have anyway and that are not truly reflective of students’ “investment.” Answering 
these questions is best done through robust data on student investment in order to model different 
scenarios and evaluate the implications of different assumptions.’ 


A lack of data makes it impossible to recommend a proper benchmark for an investment-to-earnings 
measure. However, current investment to earnings ratios should mirror those used in debt-to-earnings 
metrics, since this would put educational investments on similar footing regardless of whether they 
were made through borrowing or out-of-pocket spending. Ultimately, this paper recommends starting 
with a debt-to-earnings rate tied to the 20 percent discretionary standard, while accumulating data to 
construct an investment-to-earnings rate. 


Earnings Growth 


Earnings growth metrics capture the difference between what graduates earned before and after 
completing a program to assess whether they received an income boost after leaving school. For 
example, the “Salary Surfer” tool constructed by the California Community College system shows, 
by program, earnings two years before entering a program and then two- and five-years after 
completing.® 


For earnings growth measures to be more effective they need to be looking at populations of 
students who had earnings beforehand. Otherwise, any amount of growth will look impressive if 
the baseline is someone who has never worked before. This means a measure of earnings growth 
is likely to be more useful at community colleges or institutions that enroll large numbers of adult 
students with previous work experience than at highly selective bachelor’s degree programs where 
most students come straight from high school. 


One other challenge with this measure is that it cannot capture the value of programs that help 
someone avert job loss or move into a role with similar earnings but with other non-economic 
benefits (e.g., shift from retail to office environment). Apart from some state data systems, there 
are also no comprehensive data sources that include pre- and post-college earnings, nor clear 
benchmarks for what growth expectations should be. Thresholds for growth expectations should 
ultimately vary by credential level and be based upon actual earnings data. 


Despite the challenges, earnings growth measures have value. Particularly for students who are 
currently working, they can help someone understand how much more money they may earn as 
a result of the credential—assisting in assessments of whether that pay raise is sufficient given a 
program’s price. 


MEASURING EARNINGS AT THE PROGRAM- AND 
INSTITUTION-LEVEL 


Earnings of graduates only should be considered when examining program-level outcomes. Why? 
Evidence shows that earnings of non-completers are similar across program types while graduates 
have very different results depending on the program attended.® In addition, programmatic measures 
of earnings that include all students require determining which program a non-completer attended,’ a 
decision that is far from straightforward at colleges that do not require students to apply to a specific 
program or declare a major right away. 


However, earnings measures at the institution-level should 

consider the outcomes of both graduates and non- G6 
graduates. These earnings measures should be reported 

separately for graduate and undergraduate students, and 

ideally by credential level as well. Including non-completers 

in institution-level earnings measures is more fitting given 

that these students’ outcomes are a concern that should be 

shared across the entire school. 


Earnings measures should 
be reported separately for 
graduate and undergraduate 
students, and ideally by 
credential level as well. 


Undergraduate programs should be defined by the level of credential—including certificates of 
different lengths, associate’s, and bachelor’s degrees—and the meta-major, or broad category of 
fields of study that combines similar majors, and notably treats all liberal arts programs as a single 
category.® This approach helps to avoid sample size issues created by separating students into 
each individual major, and reflects the interdisciplinary nature of many undergraduate liberal arts 
programs.° At the graduate level, the combination of credential level and four-digit Classification of 
Instructional Program (CIP) code is appropriate for defining programs. 


OPTIONS FOR USING LOAN REPAYMENT RATES TO MEASURE 
POSTSECONDARY VALUE 


Repayment rates offer additional and important information that is crucial for measuring ROI (Table 
A2). This approach sets a higher bar for success—paying down loan principal—than avoiding the 
worst possible student loan outcome of default. This section first discusses how to best define a 
repayment rate and then examines the importance of looking at repayment rates by race/ethnicity 
and gender and completion status when considering the value of postsecondary education. 


Defining a Repayment Rate 


There are two common approaches to defining a repayment rate.'° The first is to categorize 
borrowers who have not defaulted on their loans and have paid down at least $1 of their original 
principal balance as successfully repaying. This is the definition used by the College Scorecard 

and is similar to the 2011 version of the gainful employment rule."' In 2018, Representative Virginia 
Foxx (R-NC) proposed a different measure of repayment when she was chairwoman of the House 
education committee.’ Instead of looking at the share of borrowers paying down their principal 
balance, the new repayment rate proposal would consider borrowers as successfully repaying their 
loans if they were either in active repayment and current on their loans (defined as less than 90 days 
late) or were pausing their payments to attend school or serve in the military.'* 


This paper recommends an expanded version of Rep. Foxx’s repayment rate that would consider a 
borrower to have a positive repayment status if they were current or less than 90 days late, had paid 
off the loan in full, were pausing payments to enroll in a postsecondary program, serve in the military, 
or pursue other service commitments such as the Peace Corps, or seeking teacher loan forgiveness. 
Any borrower who defaulted during the measurement window would not be considered as 
successfully repaying their loans even if they later returned their loan to good standing. This measure 
is more than just the inverse of a default rate—it requires borrowers to be staying current or engaging 
in specific loan pauses. Because this test for success is potentially easier to meet than paying down 
$1 of principal, institutions should be required to demonstrate higher levels of performance than the 
45 percent threshold proposed by Rep. Foxx. Table 2 models the potential effects of a few different 
thresholds. It suggests drawing a threshold of at least 50 percent and no higher than 67 percent 
given the steep decline in results above that level. 


Table 2. Share of Institutions Passing a Status-based Repayment Rate Based upon 


Different Thresholds 


I CCT CC 


Public 4-year 100% 98% 80% 53% 
For-Profit 4-Year 101 96% 84% 48% 39% 
Private Nonprofit 4-Year 1,179 99% 98% 87% 70% 
Public 2-Year 633 94% 80% 16% 6% 

For-Profit 2-Year 247 74% 53% 11% 6% 

Private Nonprofit 2-Year 114 87% 75% 46% 32% 
Public Less Than 2-Year 290 86% 74% 24% 13% 
For-Profit Less Than 2-Year 1,120 71% 57% 23% 11% 
Private Nonprofit Less Than 2-Year We) 77% 65% 47% 38% 


Historically Black Colleges and Universities (HBCU) 90 88% 68% 6% 1% 
Predominantly Black Institutions (PBI) 64 83% 63% 9% 0% 
Alaska Native and Native Hawaiian Serving 8 100% 88% 63% 13% 
nstitutions (ANNHI) 

Tribal Colleges and Universities (TCU) 3 67% 67% 0% 0% 
Native American Serving, Nontribal Institutions 22 86% 77% 18% 5% 
(NASNTI) 

Asian American and Native American Pacific 56 98% 93% 61% 39% 
slander-Serving Institutions (AANAPISI) 

Hispanic Serving Institutions (HSI) 299 95% 87% 43% 23% 


Note: A status-based repayment rate considers borrowers a success if they have paid off their loans, are less than 90 
days late, or on an in-school or military deferment. Borrowers entering repayment in FY2012, tracked for three years. 


Source: Author’s analysis of College Scorecard data and information obtained from a Freedom of Information 
Act request, which is available at Ben Miller, “How You Can See Your College’s Long-Term Default Rate,” Center 
for American Progress, August 30, 2018, https://www.americanprogress.org/issues/education-postsecondary/ 
news/2018/08/30/457296/can-see-colleges-long-term-default-rate/. 


A status-based measure is also less potentially misleading than one based on principal reduction. 
The primary areas of concern related to principal reduction are: 


1. Borrowers whose balances grow for legitimate reasons could be labeled as failures. A borrower 
who enters repayment and then goes to graduate school or re-enrolls could see their loan 
balance grow due to interest accumulation while they are in school, even if they made all 
required payments.'* 


2. Debt from multiple colleges may affect repayment. A borrower who accumulates $30,000 in debt 
in equal amounts from three colleges might fail a repayment test, even if they could have repaid 
the $10,000 amount borrowed from any individual college." 


3. Income Driven Repayment (IDR) options can lower repayment rates. A borrower who uses IDR 
could appear to fail the repayment test early on in repayment if their earnings are low, but may 
later begin to pay off principal.'* Considering borrowers in IDR whose payments are not sufficient 
to offset accumulating interest could also discourage institutions from getting borrowers to use 
this safety net option. 


A status-based repayment rate addresses most of the drawbacks of a measure based on balance 
paydown, such as the one used on the College Scorecard. Because it only focuses on whether 
borrowers are current or pausing payments for a limited number of valid reasons, issues related to interest 
accumulation or how debt levels affect payment strategy become less relevant. At the same time, this 
approach could be considered too forgiving because it potentially allows for institutions to encourage 
borrowers to adopt IDR plans that are classified as a repayment success. Concerns about IDR can be 
captured through a measure of how often borrowers use these options or a debt-to-earnings rate. 


Table 3 models the difference between a principal reduction and status-based repayment rate.” The table 
shows that a $1 reduction in principal test is tougher than a status-based rate at the same threshold. This 
is unsurprising because there are many repayment options that result in overall balance increases despite 
borrowers making on-time payments. It is also why a status-based rate needs a higher threshold. 


Table 3. Comparing Measures of Repayment 


# At or Over Under 45% 
Schools EW Mol MC loLeye Ml Mm CLefete im Mey-11) 
Loan Status, | Status, 


bYolo] g-Yor-1 Ke | Toto) -Yor-1 ce | 
Repayment Repayment at 
Under 45% or Over 45% 


Public 4-year 544 0% 15% 0% 85% 
For-Profit 4-Year 79 4% 59% 0% 37% 
Private Nonprofit 4-Year 1,036 1% 12% 0% 87% 
Public 2-Year 617 6% 62% 1% 31% 
For-Profit 2-Year 187 27% 52% 1% 20% 
Private Nonprofit 2-Year 99 12% 33% 0% 55% 
Public Less Than 2-Year 282 13% 50% 0% 37% 
For-Profit Less Than 2-Year 924 24% 51% 0% 25% 
Private Nonprofit Less Than 2-Year 70 21% 30% 1% 47% 
a a a Pe ie ne iba 

Historically Black Colleges and Universities (HBCU) 87 13% 79% 0% 8% 

Predominantly Black Institutions (PBI) 64 17% 73% 0% 9% 

Cae Hawaiian Serving 8 0% 25% 0% 75% 
Tribal Colleges and Universities (TCU) 8 33% 67% 0% 0% 

Ne Serving, Nontribal Institutions 29 14% 45% 0% A1% 
Seo ee Co ed 7 
Hispanic Serving Institutions (HSI) 298 4% 52% 0% 43% 


Note: A status-based repayment rate considers borrowers a success if they have paid off their loans, are less than 90 
days late, or on an in-school or military deferment. Three-year repayment rates based upon a borrower’s loan status or a 
reduction of at least $1 in principal. 


Source: Author’s analysis of College Scorecard data and information obtained from a Freedom of Information 
Act request, which is available at Ben Miller, “How You Can See Your College’s Long-Term Default Rate,” Center 
for American Progress, August 30, 2018, https://www.americanprogress.org/issues/education-postsecondary/ 
news/2018/08/30/457296/can-see-colleges-long-term-default-rate/. 
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Examining Repayment Rates by Race/Ethnicity and Gender 


In order to have a more nuanced understanding of ROI, 

repayment rates need to be examined by race/ethnicity and 46 
gender. Survey data from the National Center for Education 
Statistics shows that Black borrowers experience lower 

rates of repayment than other students. By 2015, the typical 

Black borrower who entered college in 2003-04 owed 113 

percent of what they originally borrowed, whereas White 

and Latinx borrowers respectively owed 65 and 83 percent 

of their original debt load."® 


In order to have a more 
nuanced understanding of 
ROI, repayment rates need to 
be examined by race/ethnicity 
and gender. 


Regardless of sector, roughly 60 percent of Black borrowers owed more than they originally 
borrowed over this same time period.'? These results are driven partly by the fact that half of Black 
borrowers in this sample defaulted within 12 years of entering college.”° 


The differences in principal reduction repayment by gender are smaller. On average, men have 
repayment rates about 1 to 5 percentage points higher than women, depending on the sector.*! The 
median difference by institution is 1 percentage point everywhere except public four-year institutions. 
However, the gap may be worse because women have higher completion rates than men and 
completers tend to have better repayment rates. 


Examining Repayment Rates for Completers and 
Non-Completers 


To fully understand the value of postsecondary education, repayment rates must be examined 

for both completers and non-completers separately. There is a 20-percentage point difference 

in principal reduction repayment rates between completers and non-completers, a result that is 
consistent across sectors.** Table 4 shows this difference by sector. 


Table 4. Differences in Three-year Repayment Rates by Completion Status 


Public 4-year 60% 73% 48% 22% 
For-Profit 4-Year 29% 45% 24% 21% 
Private Nonprofit 4-Year 60% 715% 47% 20% 
Public 2-Year 38% 58% 33% 22% 
For-Profit 2-Year 29% 40% 21% 17% 
Private Nonprofit 2-Year 36% 51% 28% 21% 
Public Less Than 2-Year 34% 53% 29% 21% 
For-Profit Less Than 2-Year 32% 38% 21% 17% 
Private Nonprofit Less Than 2-Year 23% 35% 14% 17% 
Total 46% 62% 36% 20% 


Note: Only includes colleges that have data on both completers and non-completers. Borrowers entering repayment in 
2013 and 2014. 


Source: Author’s analysis of data from the College Scorecard. 
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THE CHALLENGE OF USING PUBLICLY AVAILABLE DATA TO 
MEASURE ROI 


While there are a number of challenges associated with 

using publicly available data to measure postsecondary G6 
value, two are particularly problematic: limited data 

available for non-completers and limited data on earnings 

and repayment rates by race/ethnicity. 


While there are a number of 
challenges associated with 
using publicly available data 
to measure postsecondary 
value, two are particularly 
problematic: limited data 


Limited Data on Non-Completers 


Students who attend but do not finish college represent available for non-completers 
a challenge for an earnings measurement, especially at and limited data on earnings 
the programmatic level. Non-completers can stop out at and repayment rates by 
different points, which likely affects the extent to which their race/ethnicity. 


earnings will be influenced by their educational experience. 
It also makes it difficult to assign a student to a particular 
program since, in many cases, students leave school before officially declaring a field of study. 


Given all these challenges, it is better to address the issue of non-completers through more specific 
measures of graduation or completion rates. This rightfully holds institutions responsible for low 
levels of student completion, but avoids the more complex question of what role an institution 

plays in the earnings of someone who enrolled only for a short time. This approach also makes it 
possible to consider multipole outcomes from non-completion, such as students who transfer from a 
community college to a four-year institution before completing a degree. 


Importantly, the College Scorecard does not currently differentiate earnings for completers versus 
non-completers.® In the context of these data limitations, an earnings growth measure will better 
reflect value for non-completers than investment to earnings. That’s because students who stopped 
out after a brief period of enrollment would have relatively low levels of investment, making earnings 
from even minimum wage work appear to outweigh the costs of their education. Earnings growth for 
non-completers should be limited to individuals who earned at least six credits (or a minimum share 
of a program’s total required credits) because students who finished very few courses might have 
distorted results. 


Limited Data on Earnings Outcomes and Repayment Rates 
by Race/Ethnicity 


There are well-known differences in earnings by race/ethnicity. A study by the Economic Policy 
Institute notes that the racial wage gaps across education levels have increased from 2000 to 2018, 
and that a Black college graduate earns about 80 percent of that of a White college graduate and a 
Latinx graduate earns roughly 85 percent.?° 


b This paper was written in 2019 before the College Scorecard added data on completers only at the program level. 
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Unfortunately, data on racial earnings differences by institution, program, and degree level are limited 
to just a few institutions and not available nationally through the College Scorecard. Also, national 
data show gaps in employment rates by race even among individuals with similar educational 
attainment, meaning that measures, such as those on the College Scorecard, that only compare 
earnings of working adults may understate racial earnings gaps.** 


Better national earnings data that are disaggregated by race/ethnicity are necessary to evaluate 
whether students’ returns to their educational investment are equitable. Better data would also 
allow for identifying programs that pay off for students of color, or those that help limit disparities in 
earnings across these groups. 


Similar to earnings outcomes, ED does not publish institution-level data on repayment rates by 
race/ethnicity. Moreover, ED does not produce student-level data for researchers to construct other 
repayment definitions or analyze status-based rates by completion status, race, or gender. It also has 
not published data on how IDR usage or enrollment at multiple institutions affects repayment rates. 


CONCLUSION 


This paper considers measures of ROI for postsecondary education using indicators related to 
earnings and student loan debt and recommends using both to assess postsecondary value. 

Based on this assessment, the following value measures should be reported for all graduates at the 
program and credential level: investment to earnings ratio, which captures the economic return a 
program produces for the investment students make; and earnings growth, which ensures earnings 
are not just a function of prior employment. Both of these indicators should be disaggregated by 
race/ethnicity and gender. 


The following four principles guide the use of earnings in a ROI calculation: 


1. Earnings measures should include only students who graduate, with exclusions for students who 
have re-enrolled or are serving in the military during the measurement year. Students who leave 
without a degree may stop out at different times, which makes it hard to create fair comparisons. 
Research on the contribution of specific institutions to the earnings of someone who did not 
finish a degree is also less established. The issue of non-completion is a crucial element of value 
and should be handled by other indicators. 


2. Earnings are better in context. The share of graduates exceeding a minimum earnings threshold, 
or a comparison of student’s debt and earnings levels, can provide richer insights than average 
or median earnings alone. That’s because certain metrics can be strongly influenced by the mix 
of majors at a particular school, and in many cases reflects a very narrow definition of economic 
success by ignoring the social value of higher education. 


3. Earnings are better suited to program-level analyses. Multiple classification schemes exist for 
grouping individual programs into similar categories; and in choosing between these, it is critical 
to consider the need for aggregation to protect students’ privacy while conducting subgroup 
analysis, while still creating meaningful categories so that important differences between fields 
do not get hidden. A program-level approach balances these considerations. At the institution- 
level, the share of students with earnings above a minimum bar is more fitting. 


ate 


4. Earnings should be measured at multiple points in time. Looking at earnings of graduates just 
one year after finishing school may be too soon since students may take more than a few 
months to get on their feet. Instead, multiple measurement points should be used, starting 
with three years out, to mirror common measurement windows for loan repayment and default. 
Additional measures of earnings five and ten-years after graduation should be used to provide 
more detail on a student’s long-term employment outcomes.?° 


In addition, for all student loan borrowers at the institution a status-based repayment rate is 
needed to judge what share of individuals are current on their loans as an initial indicator of whether 
and to what extent student borrowers may have problems with their loans. This approach sidesteps 
significant problems with measuring repayment through principal balance reduction. Specifically, 

a balance-based repayment measure raises concern due to the variety of repayment options and 
the effects of debt from multiple institutions. In contrast, a status-based repayment measure defines 
successful repayment as borrowers who have never defaulted and who have paid their loans in 

full, are less than 90 days delinquent, or are pausing payments for acceptable reasons. A passing 
threshold on a status-based measure should be between 50 percent and 67 percent based upon 
initial data analysis. This measure should be evaluated at the institutional level, separated by 
credential level, measured at three, five, and ten years into repayment, and it should include all 
students regardless of completion status and report results of parent borrowers by level. Moreover, 
while overuse of IDR can be a policy concern, it would be better tackled through a more direct 
measure such as IDR usage or debt-to-earnings. 


As with any research project, more questions were raised than answered. As a result, the 
Commission should consider the following to better capture ROI: 


1. How to account for students who are not employed—and may not be in the labor force—during 
the measurement year.?° 

How to convert non-loan payments into an investment similar to a loan for comparison purposes. 
How to account for time spent by students in measures of investment. 


How to incorporate students who pursue advanced degrees in measures of post-school earnings. 


Of & 


The best ways to construct and evaluate earnings measures for students who do not complete 
their programs of study. 


6. How debt from multiple colleges affects loan repayment performance. 


7. How feasible it is from a systems standpoint to assess repayment in terms of on-time payments 
made versus an overall delinquency rate at the end of a tracking period. 


8. The proper way to assess over-reliance on IDR. 

9. The demographic correlations of repayment rates and race/ethnicity. 

In sum, these earnings and student loan debt measures capture important elements of ROI, yet 
they should be used in concert with other indicators to provide important context, especially related 
to completion. A better understanding of completion rates of institutions will account for situations 


where an institution or program may appear to be successful based on its graduates but has large 
numbers of students who do not finish. 
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APPENDIX 


Table A1. Federal and State Measures of Earnings 


Purpose 


Description 


Earnings 
Source 


Location 


Unit of 
Analysis 


Coverage 


Students 
Captured 


Timeframe 


Thresholds 


Accountability 


The ratio of annual payments on student 
loan debt compared to two measures 

of the earnings of graduates. Measured 
both on an annual basis and by looking at 
earnings over 150% of the poverty level. 


Social Security Administration 


2014 Gainful employment regulation 


Programs within institutions 


All programs at private for-profit 
colleges and non-degree programs at 
public and private nonprofit colleges. 


All students who received federal 
financial aid and graduated. 


The third and fourth year after 
graduating. 


8% of annual earnings, 20% of 
discretionary earnings 


Federal Measures of Earnings 


Disclosure 


The average or 50th percentile 
earnings of students who 
received financial aid within 

a set period of time of first 
entering college. 


U.S. Treasury 


College Scorecard 


Institutional* 


All colleges 


All students who received 
federal financial aid and 
are working, regardless of 
completion status. 


Reported six, eight, and 10 


years after first entering college. 


N/A 


Disclosure 


Percentage of students earning more 
than a set dollar amount, typically 
expressed as either a percentage of 
the poverty level or the earnings of 
high school graduates. 


U.S. Treasury 


College Scorecard 


Institutional* 


All colleges 


All students who received federal 
financial aid, regardless of 
completion status. 


Reported six, eight, and 10 years 
after first entering college. 


$25,000 or $28,000 


Disclosure 


The 25th, 50th, and 75th percentile 
earnings for graduates at a small 
number of institutions who earned 
a bachelor’s degree or a graduate 
credential, broken down by major. 


U.S. Census’ Longitudinal Employer- 
Household Dynamics (LEHD) data 


U.S. Census Beta tool 


Programs within institutions and 
institutions 


All public colleges in Colorado, 

the University of Texas System, the 
University of Michigan-Ann Arbor, and 
the University of Wisconsin-Madison. 


All students who graduated at 
certain schools and in certain 
degree levels at those schools, 
except for those with no earnings 
for half the year or who earned less 
than the minimum wage. 


One, five, and 10 years after 
graduation. 


N/A 


Note: *Since the time this was written, the Department of Education updated the College Scorecard to include program-level earnings. 
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State Measures of Earnings Examples 


Purpose 


Description 


Earnings 
Source 


Location 


Unit of 
Analysis 


Coverage 


Students 
Captured 


Disclosure 


Median salary two years 
before and two and five 

years after finishing a degree 
or certificate program at a 
California Community College, 
broken down by major. 


State unemployment 
insurance database 


California Community 
Colleges 


Programs aggregated across all 
California community colleges 


Students who earned a 
"terminal" degree or certificate 
at a California community 
college, meaning they did not 
transfer and are not enrolled 
and who are covered by state 
unemployment insurance 
records. 


Students who have a Social 
Security number, who are not 
unemployed, self-employed, 
federal employees, in the 
military, or living out of state. 


Disclosure 


Median salary three years 
after graduating broken down 
by institution and major. 


State unemployment 
insurance database 


California Community 
Colleges’ Chancellor's Office 


Programs within institutions 
and institutions 


Students who earned a 
"terminal" degree or certificate 
at a California community 
college, meaning they did not 
transfer and are not enrolled 
and who are covered by state 
unemployment insurance 
records. 


Students who have a Social 
Security number, who are not 
unemployed, self-employed, 
federal employees, in the 
military, or living out of state. 


Disclosure 


Salary 18 months, three 
years, and five years after 
completing by program 
and institution at Virginia 
public and private nonprofit 
colleges. 


State unemployment 
insurance database 


State Council of Higher 
Education for Virginia 


Programs within institutions 
and institutions 


Students who graduated 
from a public or private 
nonprofit college in Virginia, 
who have a Social Security 
number and who are not 


unemployed, self-employed, 


federal employees, in the 
military, or living out of 
state. 


Students who have a 
Social Security number, 
who are not unemployed, 
self-employed, federal 
employees, in the military, 
or living out of state. 
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Disclosure 


Average earnings for one 
quarter of recent graduates 
of bachelor's degree or 
higher from Florida public 
colleges, broken down by 
institution and major. 


State unemployment 
insurance database 


Florida Department of 
Education 


Programs within institutions 
and institutions 


Students who earned 

a bachelor's degree or 
graduate degree from a 
public college in Florida, 
who have a Social Security 
number and who are not 


unemployed, self-employed, 


federal employees, in the 


military, or living out of state. 


Students who have a Social 
Security number, who are not 
unemployed, self-employed, 


federal employees, in the 


military, or living out of state. 


Funding 


Institutions receive part 

of the difference in tax 
revenues collected 
between graduates from 
an institution and the 
minimum wage, measured 
over a five-year average. 


State unemployment 
insurance database 


Texas Higher Education 
Coordinating Board 


Institutions 


Students who graduated 
from a Texas State 
Technical College and 
who can be tracked by 
the Texas Workforce 
Commission. That means 
students need a Social 
Security number and 
cannot be self-employed, 
federal employees, in the 
military, or living out of 
state. 


Students who have a 
Social Security number, 
who are not unemployed, 
self-employed, federal 
employees, in the military, 
or living out of state. 


State Measures of Earnings Examples 


Timeframe 


Thresholds 


Source 


Two years before entering a 
program, as well as two and 
five years after finishing a 
program. 


N/A 


https://salarysurfer.cccco.edu/ 
SalarySurfer.aspx 


Three years after finishing, 
aggregated across eight 
years. 


N/A 


https://datamart.cccco.edu/ 
datamart.aspx 


Graduates 18 months, 

five years, and eight years 
after finishing, with longer- 
term data available at the 
program but not institutional 
level. Data are aggregated 
across five years. 


N/A 


https://research.schev.edu/ 
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The fourth quarter of the 
calendar year for the year 
that students graduated 
(e.g., Q4 2017 for 2016-17 
graduates). Data are also 
broken down by the number 
of students in earnings 


A five-year average of 
recent graduates. 


ranges. 

N/A Earnings over the 
minimum wage 

https://www.fldoe.org/ https://www.tstc.edu/ 


accountability/fl-edu- 
training-placement-info- 
program/ 


Table A2. Repayment Rates 


2011 Gainful 
Employment 


2014 Gainful 
Employment 


(exe) |(-Ye[-) 


Scorecard 


Borrower 
Defense Rule 


PROSPER Act 


Student 
Protection and 
Success Act 


Lamar Alexander 
Proposal 


Usage 


Level of 
Analysis 


Unit of 
Analysis 
(Dollar or 
Borrower) 


Numerator 


Denominator 


Threshold 


Number of 
Cohorts in 
the Measure 


Accountability 


Programmatic 


Dollar 


Sum of all original 
loan balances 
where a borrower 
paid at least $1 of 
principal at the time 
of measurement. 


Sum of all original 
loan balances for 
borrowers who 
entered repayment 
during the cohort 
measurement 
period. 


35% for three 
consecutive years 
while also failing 
two measures of 
debt relative to 
earnings. 


Two 


Disclosure 


Programmatic 


Dollar 


Sum of all original 
loan balances 
where a borrower 
paid at least 

$1 of principal 

at the time of 
measurement. 


Sum of all original 
loan balances for 
borrowers who 
entered repayment 
during the cohort 
measurement 
period. 


Two (four for small 
programs) 


Transparency 


Institutional 


Borrower 


Number of 
borrowers who 
had not defaulted 
and paid down at 
least $1 of their 
original principal 
balance. 


Number of 
borrowers who 
entered repayment 
during the cohort 
measurement 
period. 


N/A 


Two 


Disclosure and 
Warnings 


Institutional 


Borrower 


N/A 


N/A 


The median 
borrower must 
have paid off their 
loan or reduced 
their original 
principal balance 
by at least $1. 


Two 
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Accountability 


Programmatic 


Borrower 


Number of borrowers 
in a positive repayment 
status, defined as 
having paid off the 
loan, being less 

than 90 days late, or 
pausing payments 
because they were 
enrolled back in school 
or serving in the 
military. 


Number of borrowers 
who entered repayment 
during the cohort 
measurement period. 


Under 45% for two 
consecutive years. 


One 


Accountability 


Institutional 


Borrower 


Borrowers who 
have not defaulted 
and paid at least 
$1 of their original 
loan principal 
balance. 


Number of 
borrowers who 
entered repayment 
during the cohort 
measurement 
period. 


Equal to or less 
than 15% for three 
consecutive years. 


One 


Accountability 


Programmatic 


Borrower 


“Whether students 
are actually paying 
off their loans.” 


Number of 
borrowers who 
entered repayment 
during the cohort 
measurement 
period. 


Unstated 


Unstated 


Years Into 
Repayment 
Measured 


How is 
Income- 
Driven 
Repayment 
Handled? 


Source 


2011 Gainful 
Employment 


Three and four 
years 


Up to 3 percentage 
points of original 
loan balances on 
income-driven 
repayment can 

be included in the 
numerator, even 

if they are not 
reducing principal. 


https://www. 
federalregister.gov/ 
documents/2011/ 
06/13/2011-13905/ 
program-integrity- 
gainful- 
employment-debt- 
measures 


2014 Gainful 
Employment 


Three and four 
years 


No special 
treatment. 


https://www. 
federalregister.gov/ 
documents/2014/ 
10/31/2014- 
25594/program- 
integrity-gainful- 
employment 
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Scorecard 


i, &, &, alnel 
7 years into 
repayment 


No special 
treatment. 


https:// 
collegescorecard. 
ed.gov/data/ 


Borrower 
Defense Rule 


No special 
treatment. 


https://www. 
federalregister. 
gov/ 
documents/2016/ 
11/01/2016- 
25448/student- 
assistance- 
general- 
provisions-federal- 
perkins-loan- 
program-federal- 
family-education- 
loan 
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PROSPER Act 


Three years 


No special treatment, 
but changes terms 
of IDR plans so all 
borrowers must 
make at least a $25 
payment. 


https://www.congress. 
gov/bill/115th- 
congress/house- 
bill/4508/text#toc-HF2 
FEBO04E73A415B974 
B56B6C7D121DF 


Student 


Protection and 
Success Act 


Three years 


No special 
treatment. 


https://www. 
congress.gov/ 
bill/116th- 
congress/senate- 
bill/1525 


Lamar Alexander 
led ce) eXey-¥- 1 | 


Unstated 


Unstated 


https://www.aei.org/ 
wp-content/uploads/ 
2019/01/190205 
-Senator-Lamar- 
Alexander-event- 
transcript.pdf 
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Programs with internship or residency components should define their measurement windows by the end of the residency. Students 
enrolled in college or serving in the armed forces at any point during the year of measurement of should be excluded from the 
numerator and denominator. 

The inclusion or exclusion of non-working individuals matters a lot for earnings assessments. For example, the College Scorecard 
shows that between 10 and 20 percent of students who entered in 2004-05 or 2005-06 and received financial aid were not working 
and not enrolled eight years after entry. Excluding non-workers thus significantly alters the share of students at less-than-two-year for- 
profit colleges who earned over $25,000 from 38 percent to 47 percent. 


Share of Students Median % of students | Change in Median 
that are not working earning over $25k % of students 


Vite Mikel m-Vilce)|(-Ye} earning over $25k 
by excluding non- 
working individuals 


Public 4-year 10% 73% 7% 

For-Profit 4-Year 17% 64% 11% 

Private Nonprofit 4-Year 11% 73% 7% 

Public 2-Year 17% 56% 11% 

For-Profit 2-Year 19% 52% 10% 

Private Nonprofit 2-Year 17% 59% 10% 

Public Less Than 2-Year 18% 54% 11% 

For-Profit Less Than 2-Year 22% 36% 9% 

Private Nonprofit Less Than 2-Year 22% 50% 16% 
Sa Ee | eg | | 

Historically Black Colleges and 12% 50% 6% 

Universities (HBCU) 

Predominantly Black Institutions 17% 50% 9% 

(PBI) 

Alaska Native and Native 16% 64% 12% 

Hawaiian Serving Institutions 

(ANNHI) 

Tribal Colleges and Universities 26% 33% 14% 

(TCU) 

Native American Serving, 16% 57% 13% 

Nontribal Institutions (NASNTI) 

Asian American and Native 17% 66% 13% 


American Pacific Islander-Serving 
Institutions (AANAPISI) 


Hispanic Serving Institutions (HSI) 16% 59% 13% 


Note: Not working, not enrolled students affect comparisons of earnings to high school graduate 2004-06 
entry cohorts measured eight years after enrollment. 


Source: Author analysis of College Scorecard data. 
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